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Detailed Action 

1. Applicant's response filed 26 October 2010 has been entered and considered, but is not 
found convincing. Claims 1, 6, 7, 11, 16, 18-20 and 24-47 have been amended without 
introducing new subject matter. No new rejections are set forth herein; therefore, this action is 
made Final. 

Response to Applicant's Remarks 

2. 35 USC § 101 

Claims 24-29, 32, 35, 38, 41, 44 and 47 were rejected under 35 USC § 101 for failing to 
recite patentable subject matter. Applicant has since amended the independent claims remedy 
the '101 issues noted in the previous action; therefore, the previously outstanding grounds of 
rejection under 35 USC §101 are hereby withdrawn. 

3. 35 USC §11212 

Claims 1-47 were rejected under 35 USC § 1 12 for failing to particularly point out and 
distinctly claim the subject matter which applicant regards as the invention. Applicant has since 
amended the claims to remedy the indefiniteness issues noted in the previous action; therefore, 
the previously outstanding grounds of rejection under 35 USC §1 12 are hereby withdrawn. 

4. 35 USC § 103 

Claims 1-3, 6-8, 11-13 and 16-47 were rejected in view of the combination of Ishiyama 
and Kohno. Applicants traverse this rejection arguing that neither Ishiyama nor Kohno teach the 
limitation of "a first sharpness extraction unit for extracting a first sharpness amount reflecting 
the sharpness from each of the plurality of comparison images" as recited throughout the similar 
independent claims. Specifically, Applicants note that in the claims the sharpness amount is 
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extracted from the comparison or generated image while Kohno uses external environment 
information to determine a similarity between images (see Remarks, pp. 18, 10/26/2010). 
Applicants conclude that neither Ishiyama nor Kohno teach the sharpness extraction limitation. 

Examiner respectfully disagrees. Ishiyama is relied upon to teach the limitation of 
generating a comparison image (see ( J[ ( J[0045-46, where Ishiyama calculates differences between a 
projected image and a pose candidate). Applicants correctly note that Ishiyama is silent on 
further extracting sharpness from the comparison image, as claimed. However, Kohno is 
introduced as an improvement to Ishiyama's system at least because Kohno teaches the concept 
of using a weighted sharpness extraction value when determining image similarities (see Kohno 
c 110045-46). Kohno and Ishiyama are both drawn to image similarity determination systems and 
so the ordinarily skilled artisan would have had ample motivation to combine them. Such a 
combination (as originally proposed in the non-final action mailed 08/13/2010) would have 
modified Ishiyama's generated comparison image by applying Kohno's weighted sharpness 
value extraction method. The resulting combination teaches "a first sharpness extraction unit for 
extracting a first sharpness amount reflecting the sharpness from each of the plurality of 
comparison images" as recited throughout the similar independent claims. 

Therefore, the combination of Ishiyama and Kohno is understood to teach each and every 
limitation of claims 1-3, 6-8, 1 1-13 and 16-47, and the previously outstanding rejections thereto 
are maintained. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 
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(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 
(1966), that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1. Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 

4. Considering objective evidence present in the application indicating obviousness or nonobviousness. 

6. Claims 1-3, 6-8, 1 1-13, 16-47 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Ishiyama U.S. Pre-Grant Application Publication No. 2003/0035098 ("Ishiyama") in view 
of Kohno U.S. Pre-Grant Application Publication No. 2003/0161535 ("Kohno"). 

Regarding similar independent claims 1 (system), 11 (system), 16 (method), 20 (method), 
24 (program) and 28 (program) Ishiyama teaches a pose estimation system for performing object 
pose estimation by comparing an input image with a three-dimensional object model (see Figs. 
3-5 and *^0010 and 0026 for an overview of Ishiyama 's pose estimation system), the pose 
estimation system comprising: 

i. an pose candidate decision unit for generating at least one pose candidate (see Ishiyama 
Fig. 3 as described in ( j[ ( j[0040-41 where a pose estimation routine is disclosed); 

ii. a comparison image generation unit for generating, according to the generated pose 
candidate, a plurality of comparison images similar to the input image, while projecting 
the three-dimensional object model to a two-dimensional image (see Ishiyama Fig. 3 as 
described in ( H0044-45 where images are projected onto a pose candidate); 
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iii. a difference calculator for calculating a plurality of differences (see Ishiyama Fig. 3 as 
described in ( H0045-46 where differences between the projected images and the pose 
candidate are calculated); and 

iv. a determination unit for selecting a comparison image having the smallest difference 
among the plurality of differences and estimating an optimal pose based on the selected 
comparison image (see Ishiyama Fig. 3 item 310 as described in K0047 where the image 
candidate whose distance to the input image is smallest is selected). 

Ishiyama is silent on the system further comprising a first sharpness extraction unit for extracting 
a first sharpness amount reflecting the sharpness from each of the plurality of comparison 
images, and a weighted difference calculator, as claimed. However, Kohno teaches a similar 
image similarity determination system comprising using a measure of sharpness (e.g., a degree of 
camera blur when acquiring an image) in similarity determination processing (see Kohno 
c HSI0065-66, where weighted sharpness (i.e. blurring) values are used to determine image 
similarity). 

The ordinarily- skilled artisan, starting with the pose estimation system of Ishiyama, 
would have appreciated the benefits of using a weighted measure of sharpness to determine pose 
similarity, as proposed by Kohno in f][0065-66. The resulting combination of Ishiyama and 
Kohno would achieve the predictable and expected benefits of increasing the accuracy of the 
pose estimation by incorporating a measure of image sharpness. The artisan of ordinary skill 
would have been motivated to combine Kohno with Ishiyama, as proposed above, at least 
because both Kohno and Ishiyama are drawn to similar image matching systems, and so the 
similarities between Kohno and Ishiyama invite further modifications between the two. 
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Therefore, a person having ordinary skill in the image processing arts at the time of the 
invention would have found it obvious to combine the weighted sharpness values of Kohno, with 
the pose estimation system of Ishiyama to achieve the well-known and expected benefits of 
increasing the accuracy of the pose estimation by incorporating a measure of image sharpness. 

Regarding similar claims 2 (system), 12 (system), 17 (method), 21 (method), 25 
(program) and 29 (program) Ishiyama and Kohno teach the method as indicted re claim 1 above, 
but are silent on the method further comprising a second sharpness extraction unit for extracting 
a second shaipness amount reflecting the sharpness from the input image, wherein the weighted 
difference calculator calculates a plurality of weighted differences by weighting the difference 
between the second sharpness amount of the input image and the first sharpness amount of each 
of the comparison images to the difference between the input image and the comparison image. 
However, official notice is taken to note that the uses and benefits of using a second sharpness 
extractor are well known and expected in the image processing arts (and Kohno already teaches a 
first sharpness extractor, as indicated re claim 1 above). It would have been obvious to the 
artisan of ordinary skill at the time of the invention to use the results from a second sharpness 
extractor within the system of Ishiyama and Kohno as described re claim 1 above, to achieve the 
known and expected uses and benefits of increasing the accuracy of the resulting pose 
estimation. 

Regarding similar claims 3 (system), 8 (system), 13 (system), Ishiyama and Kohno teach 
the method as indicted re claim 1 above, but are silent on the method further comprising wherein 
the weight becomes higher as the sharpness of the image becomes higher in the weighted 
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difference calculation. However, official notice is taken to note that the uses and benefits of 
using a variable weight are well known and expected in the image processing arts (and Kohno 
already teaches a weighted first sharpness value, as indicated re claim 1 above). It would have 
been obvious to the artisan of ordinary skill at the time of the invention to use a variable 
weighted sharpness result within the system of Ishiyama and Kohno as described re claim 1 
above, to achieve the known and expected uses and benefits of increasing the accuracy of the 
resulting pose estimation. 

Regarding similar claims 6 (system), 7 (system), 26 (program) and 27 (program), 
Ishiyama and Kohno teach the method as indicted re claim 1 above, but are silent on the method 
further comprising wherein the determination unit further performs object comparison by 
comparing the minimum weighted difference of the estimated optimal pose with a predetermined 
threshold value. However, official notice is taken to note that the uses and benefits of using a 
predetermined threshold in pose estimation are well known and expected in the image processing 
arts. It would have been obvious to the artisan of ordinary skill at the time of the invention to 
use a predetermined threshold within the system of Ishiyama and Kohno as described re claim 1 
above, to achieve the known and expected uses and benefits of increasing the accuracy of the 
resulting pose estimation. 

Regarding similar claims 18 (method) and 19 (method), Ishiyama further teaches 
selecting a comparison image having the smallest weighted difference among the plurality of 
weighted differences; and estimating an optimal pose based on the selected comparison image 
(see Ishiyama Fig. 3 item 310 as described in ( |0047 where the image candidate whose distance 
to the input image is smallest is selected). 
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Regarding similar claims 22 (method) and 23 (method), Ishiyama further teaches 
difference calculation method according to claim 20, further comprising: performing comparison 
by comparing the plurality of weighted differences obtained by the calculation (see Ishiyama 
Fig. 3 item 310 as described in 10047 where the image candidate whose distance to the input 
image is smallest is selected). 

Regarding similar independent claims 30 (method), 36 (method) and 42 (method) 
Ishiyama teaches object pose/illumination estimation method (see Figs. 3-5 and 110010 and 
0026 for an overview of Ishiyama 's pose estimation system) for estimating at least one of the 
pose and the illumination conditions of an object (see Ishiyama Fig. 3 as described in 110040-41 
where a pose estimation routine is disclosed) by generating a two-dimensional image of the 
object while changing at least one of the pose and the illumination conditions of the object with 
the use of a three-dimensional model of the object(see Ishiyama Fig. 3 as described in 110044- 
45 where images are projected onto a pose candidate). 

Ishiyama is silent on the system wherein sharpness of the generated two-dimensional 
image is reflected in the similarity, as claimed. However, Kohno teaches a similar image 
similarity determination system comprising using a measure of sharpness (e.g., a degree of 
camera blur when acquiring an image) in similarity determination processing (see Kohno 
110065-66, where weighted sharpness (i.e. blurring) values are used to determine image 
similarity). Motivation to combine Ishiyama and Kohno to achieve the results of claim 30, 36 
and 42 can be found re similar claim 1 above. 

Regarding similar independent claims 31 (system), 37 (system) and 43 (system) Ishiyama 
teaches an image generation unit for generating a two-dimensional image of an object (see Figs. 
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3-5 and y^OOlO and 0026 for an overview oflshiyama 's pose estimation system) while changing 
at least one of the pose and the illumination conditions of the object with the use of a three- 
dimensional model of the object (see Ishiyama Fig. 3 as described in KK0040-41 where a pose 
estimation routine is disclosed); a calculator for calculating the similarity by comparing the 
generated two-dimensional image with an input image (see Ishiyama Fig. 3 item 310 as 
described in ( j[0047 where the image candidate whose distance to the input image is smallest is 
selected). 

Ishiyama is silent on the system further comprising an extraction unit for extracting 
shaipness from the generated two-dimensional image and reflecting the extracted sharpness in 
the calculation; and a determination unit for estimating at least one of the pose and the 
illumination conditions based on the calculation result of the calculator, as claimed. However, 
Kohno teaches a similar image similarity determination system comprising using a measure of 
sharpness (e.g., a degree of camera blur when acquiring an image) in similarity determination 
processing (see Kohno ( |[ ( |[0065-66, where weighted sharpness (i.e. blurring) values are used to 
determine image similarity). Motivation to combine Ishiyama and Kohno to achieve the results 
of claim 31, 37 and 43 can be found re similar claim 1 above. 

Regarding similar independent claims 32 (system), 38 (system) and 44 (system) Ishiyama 
teaches image generation processing for generating a two-dimensional image of the object (see 
Figs. 3-5 and W0010 and 0026 for an overview of Ishiyama 's pose estimation system) while 
changing at least one of the pose and the illumination conditions of the object with the use of a 
three-dimensional model of the object (see Ishiyama Fig. 3 as described in ( H0040-41 where a 
pose estimation routine is disclosed); calculation processing for calculating the similarity by 
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comparing the generated two-dimensional image with an input image (see Ishiyama Fig. 3 item 
310 as described in 10047 where the image candidate whose distance to the input image is 
smallest is selected). 

Ishiyama is silent on the system further comprising extraction processing for extracting 
sharpness from the generated two-dimensional image, and reflecting the extracted sharpness in 
the calculation, as claimed. However, Kohno teaches a similar image similarity determination 
system comprising using a measure of sharpness (e.g., a degree of camera blur when acquiring 
an image) in similarity determination processing (see Kohno 110065-66, where weighted 
sharpness (i.e. blurring) values are used to determine image similarity). Motivation to combine 
Ishiyama and Kohno to achieve the results of claim 32, 38 and 44 can be found re similar claim 1 
above. 

Regarding similar claims 33 (method), 34 (system), 35 (program), 39 (method), 40 
(system), 41 (program), 45 (method), 46 (system) and 47 (program), Ishiyama and Kohno teach a 
pose estimation system but are silent on the system further comprising wherein, if the two- 
dimensional image having the highest similarity in brightness is not, sharp the estimation to the 
input image is not employed. However, official notice is taken to note that the uses and benefits 
of varying the sharpness estimation based upon similarity are well known and expected in the 
image processing arts. It would have been obvious to the artisan of ordinary skill at the time of 
the invention to vary the sharpness estimation based upon similarity within the system of 
Ishiyama and Kohno as described above, to achieve the known and expected uses and benefits of 
increasing the accuracy of the resulting pose estimation. 
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Claim Objections 

7. Claims 4-5, 9-10, and 14-15 would be allowable if rewritten to include all the limitations 
of their respective base claims and any intervening claims. 

8. The following is a statement of reasons for the indication of allowable subject matter: 
Regarding similar dependent claims 4-5, 9-10, and 14-15 the prior art of record, alone or in 
combination, fails to fairly teach of suggest the limitations of: wherein the first and second 
sharpness amounts are defined by a ratio of a number of pixels whose edge intensity is a 
threshold value or higher to the total number of pixels, a range of brightness values, dispersion of 
brightness values, or a number of characteristic points, and wherein the first and second 
sharpness amounts are defined by an edge image or a characteristic point, as recited in the claims 

Conclusion 

9. No new grounds of rejection set forth herein; accordingly, this action is made final. 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). A 
shortened statutory period for reply to this final action is set to expire THREE MONTHS from 
the mailing date of this action. In the event a first reply is filed within TWO MONTHS of the 
mailing date of this final action and the advisory action is not mailed until after the end of the 
THREE-MONTH shortened statutory period, then the shortened statutory period will expire on 
the date the advisory action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will the statutory 
period for reply expire later than SIX MONTHS from the mailing date of this final action. 
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Contact 

10. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Steve Koziol: 

• phone (571) 270-1844, fax (571) 270-2844, or e-mail stephen.koziol @ uspto .gov . 
Typically, the examiner can be reached Monday - Friday 9:00 - 5:30 EST. For e-mail 
communications, please note MPEP 502.03, which states, in relevant part, "[w]ithout a written 
authorization by applicant in place, the USPTO will not respond via Internet e-mail to any 
Internet correspondence which contains information subject to the confidentiality requirement as 
set forth in 35 U.S.C. § 122." A sample authorization form which may be used by applicant can 
be found in MPEP 502.03 section II. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Bhavesh Mehta can be reached at (571) 272-74 53. Customer Service can be reached 
at (571) 272-2600. The fax number for the organization where this application or proceeding is 
assigned is (571) 273-8300. Information regarding the status of an application may be obtained 
from the Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. Status 
information for unpublished applications is available through Private PAIR only. For more 
information about the PAIR system, see http://pair-direct.uspto.gov. Should you have questions 
on access to the Private PAIR system, contact the Electronic Business Center (EBC) at 866-217- 
9197 (toll-free), 
/srk/ 

30 December 2010 
/Jingge Wu/ 

Primary Patent Examiner, Art Unit 2624 
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